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INTRODUCTION  AND  EXECUTIVE  SUMMARY 
Research  Goals 

Injured  Warfighters  return  home  to  faee  long-term  eare  and  reeovery  in  addition  to  life 
adjustments.  Rehabilitation  serviees — sueh  as  interventions  for  traumatie  brain  injury  (TBI)- 
indueed  motor  limitations,  broken  bones,  spinal  cord  injuries,  chronic  pain,  and  amputation — 
enable  these  Warfighters  to  adjust  to  new  living  constraints  and  conditions  and,  in  many  cases, 
return  to  full  health.  While  these  services  are  readily  available  in  military  treatment  facilities 
(MTFs)  and  veterans’  affairs  medical  centers  (VAMCs),  not  all  patients  have  the  time  or  ability 
to  receive  prolonged  inpatient  rehabilitation  interventions.  Furthermore,  lengthy  inpatient 
treatments  are  costly  to  MTFs  and  VAMCs,  reducing  the  overall  number  and  types  of  services 
that  these  facilities  can  provide. 

For  these  reasons,  home -based  and  outpatient  rehabilitation  interventions  hold  great  potential  to 
improve  the  rehabilitation  of  our  Warfighters.  These  rehabilitation  interventions  enable  patients 
to  continue  with  their  daily  lives  during  rehabilitation.  Patients  can  perform  professional  duties; 
be  with  family;  and  be  social  with  friends  around  the  schedule  of  their  rehabilitation  practice, 
and  all  of  these  functions  enable  patients  to  better  adjust  to  life  changes  that  follow  injury. 
Home-based  and  outpatient  rehabilitation  interventions  are  accessible  to  a  wide  range  of 
patients  because  they  lower  the  time  and  travel  requirements  of  rehabilitation.  Finally,  home- 
based  and  outpatient  rehabilitation  interventions  are  less  costly  to  MTF  and  VAMC  service 
providers,  enabling  these  facilities  to  provide  a  wider  range  of  services  to  more  patients. 

The  patient  must  practice  therapeutic  exercises  regularly.  The  unfortunate  reality  of  many 
home-based  and  outpatient  therapies  is  that  the  patient  does  not  regularly  practice  therapeutic 
exercises  beyond  visits  with  the  therapist  and,  therefore,  does  not  see  significant  improvement. 
Studies  of  home-based  and  outpatient  rehabilitation  interventions  have  identified  a  number  of 
key  correlates  to  lack  of  adherence;  confusion  about  exercises;  perceptions  of  lack  of  time  to 
exercise;  forgetting  to  exercise;  perceptions  of  helplessness;  and  overall  lack  of  motivation  to 
exercise  (Jette  et  ah,  1998;  Sluijs,  Kok,  &  van  der  Zee,  1993).  Conversely,  patients  who  have 
less  confusion,  make  time  to  exercise,  remember  to  exercise,  perceive  higher  self-efficacy,  and 
report  motivation  to  exercise  adhere  more  regularly  to  rehabilitation  protocols.  In  addition  to 
these  areas  of  needed  patient  assistance,  outpatient  therapists  must  be  enabled  to  perform  their 
job  functions  of  observing  the  patient  and  directing  exercises. 

For  these  reasons,  remote  assistance  to  home-based  and  outpatient  rehabilitation  is  needed  to 
enhance  the  recovery  of  our  injured  Warfighters. 

Description  of  the  Technical  Approach 

To  address  these  issues,  we  are  developing  mobile,  virtual  enhancements  for  rehabilitation 
(MOVER),  a  mobile,  technology-enabled  home-based  rehabilitation  intervention  delivery 
system.  MOVER  features  (1)  a  mobile  application  to  provide  education,  information,  and 
scheduling  of  therapeutic  exercises;  (2)  virtual  coaches  to  guide,  mentor,  and  motivate  patients; 
(3)  COTS  input  devices  and  video  games  to  increase  patient  motivation;  and  (4)  a  web-based 
therapist  interface  to  accurately  assess  patient  adherence  and  progress. 
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Figure  1  shows  the  MOVER  Architecture.  At  the  top  left  of  the  figure,  the  Patient  interacts 
with  the  MOVER  Mobile  Application  to  perform  Exercise  Scheduling  and  obtain 
Information  and  Education  about  therapeutic  exercises.  The  Virtual  Coaches  exist  on  the 
mobile  application  and  provide  interactive  guidance  and  mentoring  about  the  rehabilitation 
process  and  therapeutic  exercises.  When  the  scheduled  time  for  the  exercises  arrives,  the 
mobile  application  reminds  the  patient,  and  the  patient  begins  an  exercise  session  with  the 
MOVER  Game  Integration,  as  shown  at  the  bottom  of  the  figure.  The  patient  uses  COTS 
input  devices,  such  as  the  Microsoft  Kinect  and  the  Wii  Balance  Board,  to  perform  therapeutic 
exercises  that  are  mapped  to  controls  of  the  Video  Game  Console  through  the  Control 
Mapping  on  Laptop,  software  running  on  an  inexpensive  PC  or  laptop. 


During  interaction  with  the  mobile  application  and  game  integration,  patient  Performance  is 
recorded  and  sent  securely  to  the  Remote  Server  and  Secure  Database,  at  the  center  of  the 
figure.  The  Therapist  reviews  this  performance  through  summarized  Progress  Reports  in  the 
web-based  Therapist  Interface,  as  shown  at  the  top  right  of  the  figure.  The  therapist  then 
creates  Therapeutic  Exercise  Assignments  to  describe  the  patient’s  therapeutic  exercises  for 
the  next  week,  and  these  assignments  are  passed  to  the  game  integration  for  implementation  the 
next  time  the  patient  begins  exercise. 
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Figure  1:  MOVER  architecture 

A  typical  use  of  MOVER  in  home-based  rehabilitation  is  as  follows.  The  patient  meets  with  the 
therapist  at  the  beginning  of  the  week  to  participate  in  a  short,  one-on-one  rehabilitation 
session.  The  therapist  assesses  the  patient,  prescribes  a  set  of  therapeutic  exercises  for  the 
week,  and  works  with  the  patient  to  determine  a  feasible  exercise  schedule.  At  the  end  of  the 
session,  the  patient  and  therapist  enter  an  exercise  schedule  for  the  week,  and  the  therapist 
enters  the  therapeutic  exercises  into  the  therapist  interface.  During  the  week,  the  patient  is 
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reminded  of  seheduled  exereises  by  the  mobile  applieation  and  motivated  by  the  virtual 
eoaehes.  The  patient  uses  the  COTS  input  deviees  to  play  the  video  games  with  the  therapeutic 
movements  specified  by  the  therapist.  During  exercise,  the  virtual  coaches  give  feedback  on 
patient  movements  and  form,  and  afterwards  the  virtual  coaches  review  the  patient 
performance.  Performance  information  is  collected  and  sent  to  the  remote  server.  At  the  next 
session,  the  therapist  reviews  the  progress  reports  with  the  patient  to  determine  next  steps  for 
treatment. 
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TECHNICAL  PROGRESS 
Progress  against  Planned  Objectives 

During  this  reporting  period,  we  focused  on  Task  3  (Virtual  Coaches  Development),  Task  4 
(Input  Device  and  Game  Integration),  and  Task  7  (Program  Management)as  presented  in  the 
Statement  of  Work  for  this  effort. 

Technical  Accomplishments  This  Period 
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Task  3:  Virtual  Coaches  Development 

Our  goal  for  Task  3  is  to  develop  virtual  coaches  to  assist  home -based  TBI  motor  impairment 
rehabilitation  therapy  through  mentoring  and  guidance.  During  the  current  reporting  period, 


During  the  previous  reporting  period,  we  designed  a  therapist  interface  to  create  and  edit 
exercises  and  schedules  for  individual  patients.  Figure  2  shows  a  mockup  of  the  exercise 
routine  editor.  On  the  left,  the  therapist  can  assign  new  exercises  to  the  patient  by  creating  them 
or  selecting  exercises  from  a  library.  In  the  middle,  exercise  information  is  editable,  including 
the  name,  purpose,  notes,  goal,  number  of  sets,  number  of  repetitions,  and  introduction  text. 
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This  exercise  helps  build  upper  body  strength  and  overall 
balance. 

Josh  has  been  progressing  steadily  on  this  exercise  over 
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Introduction 
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Stand  with  feet  shoulder  width  apart,  facing  the  camera 
Slowly  raise  one  arm  to  90  degrees 
Hold  that  pose  for  30  seconds 
Slowly  lower  the  arm  to  resting 


0@@ 


Figure  2:  MOVER  exercise  routine  editor  mockup,  exercise  Information 


Figure  3  shows  the  prototype  exercise  editor  created  from  this  early  mockup.  The  editor  allows 
the  capture  of  recorded  pose  data  from  the  Kinect  and  the  mark-up  of  that  data  to  specific 
distance  and  angle  constraints  on  exercises  within  time  segments.  For  example,  a  therapist  may 
record  a  patient  performing  leg  lift  exercises,  and  specify  that  the  knee  reach  a  90  degree  angle 
during  the  middle  of  a  repetition.  Upon  showing  this  interface  to  the  therapist  SMEs  at 
Spaulding,  we  determined  that  the  specification  of  time  segments  and  constraints  was  overly 
complex  for  the  end  user,  and  we  have  elected  to  reserve  this  tool  for  developers  to  create  new 
exercises  for  the  MOVER  system. 
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Figure  3:  Prototype  exercise  editor 

In  place  of  having  an  exercise  creation  tool  for  the  therapists,  we  have  simplified  the  process  by 
hardcoding  specific,  commonly  used  balance  exercises  into  the  system  and  enabling  the 
therapists  to  select  and  customize  pre-identified  parameters  for  these  exercises.  Figure  4  shows 
the  exercise  routine  selection  screen.  Each  exercise  has  a  custom  set  of  parameters.  For 
example.  Error!  Reference  source  not  found,  shows  the  height  setting  for  the  knee  lift 
exercise.  The  therapist  selects  the  height  that  the  patient  attempts  to  achieve  during  each  knee 
lift,  and  the  height  is  visualized  as  a  vertical  percentage  slider  and  red  bar  over  the  patient  as  a 
point  of  reference. 
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Figure  4:  Exercise  routine  selection 


Task  4:  Input  Device  and  Game  Integration 

Our  goal  for  Task  4  is  to  integrate  commercial-off-the-shelf  (COTS)  input  devices  and  COTS 
video  games  to  engage  the  patient  and  estimate  body  posture  during  therapeutic  exercises. 

During  the  current  reporting  period,  we  presented  the  current  MOVER  demo  to  our  therapist 
SMEs  at  Spaulding  Rehabilitation  Hospital,  as  shown  in  Eigure  5  and  Eigure  6.  In  this 
demonstration,  the  user  can  record  and  practice  an  exercise  with  voice  input  and  visual 
feedback.  Eigure  5  shows  a  user  (a  stand  in  for  a  patient)  recording  a  new  exercise.  The  user 
stands  in  front  of  the  camera  and  says  “go  to  recording”  into  the  wireless  headset  they  are 
wearing.  Once  in  recording,  the  user  says  “begin”,  performs  the  movements  of  this  exercise, 
and  says  “done”.  The  exercise  movements  are  saved  to  disk  for  later  practice.  Eigure  6  shows 
the  practice  of  this  exercise.  The  user  again  uses  the  voice  commands  to  enter  practice  mode. 
The  virtual  coach  appears  on  the  left  and  a  white  outline  view  appears  over  the  user  when  the 
exercise  begins.  The  white  outline  and  virtual  coach  perform  the  movement  as  recorded,  and 
the  user  follows  along,  tracked  by  a  green  skeleton.  If  the  user  gets  out  of  position,  the  out-of¬ 
place  skeleton  bones  are  colored  red. 
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Figure  5:  MOVER  prototype  screenshot,  recording 


Figure  6:  MOVER  prototype  screenshot,  practice 

Through  providing  this  early  prototype  to  our  therapist  SMEs  and  presenting  our  plans  for 
commercial  game  integration,  we  received  several  points  of  feedback,  including  the  following: 
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1.  The  voice  commands  might  not  work  well  in  our  sometimes  noisy  laboratory  test 
environment.  A  hand-held  mouse  or  clicker  would  be  easier  to  use. 

2.  The  exercise  screen  was  too  busy  with  too  much  going  on  on  the  screen.  This  could 
negatively  affect  the  ability  for  TBI  patients  with  visual  and  cognitive  deficits  to  use  the 
system. 

3.  Commercial  games,  while  high  in  entertainment  value,  are  often  not  well  tuned  to  the 
needs  of  TBI  patients,  especially  with  the  aforementioned  visual  and  cognitive  deficits. 
Instead,  the  therapists  felt  the  need  for  simpler  games  that  are  highly  customizeable. 

To  address  these  issues,  we  changed  the  design  of  the  system  in  the  following  ways: 

1.  We  added  mouse/clicker  support  for  the  menus  and  removed  the  voice  command 
requirement. 

2.  We  removed  the  virtual  coach’s  character  from  the  screen,  and  removed  as  much  of  the 
overlays  as  possible  while  still  preserving  the  core  aspects  of  the  exercises. 

3.  We  added  a  small  suite  of  simple  exercise  games  tuned  to  motion  and  balance  disorder 
patients.  We  made  these  games  highly  customizable  to  enable  therapists  to  tune  each 
game  to  the  capabilities  of  individual  patients. 

Significant  Changes  to  Technical  Approach  to  Date 

There  have  been  no  significant  changes  to  the  technical  approach  to  date. 

Deliverables  Submitted  This  Period 

During  the  current  reporting  period,  we  submitted  1  quarterly  project  status  updates  to  the  JPC- 

1  council. 

Milestones  Reached/ Achieved  During  This  Period 

The  progress  under  the  above  tasks  details  the  milestones  achieved  this  period. 


PROJECT  PLANS 

Specific  Objectives  for  Next  Period 

■  Develop  the  exercises  and  virtual  coach  feedback  based  on  our  new  designs. 


ISSUES  OR  CONCERNS 

We  have  no  technical  issues  or  concerns  at  this  time.  As  of  the  time  of  writing,  the  option 
period  is  yet  unfunded,  and  we  are  therefore  seeking  methods  to  run  the  initial  clinical  trials  as 
described  in  the  option  period. 


EXPENDITURES 
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Total  Contract  Amount 

$767,388.00 

Costs  Incurred  this  reporting 
period 

$122,691.47 

Costs  Incurred  to  Date 

$496,530.88 

Estimated  %  to  eompletion 

62.5% 

